Persuasion modeling is a key building block for conversational agents. Existing works in this direction are limited to analyzing textual dialogue corpus. We argue that visual signals also play an important role in understanding human persuasive behaviors. In this paper, we introduce the first multimodal dataset for modeling persuasion behaviors. Our dataset includes 199 dialogue transcriptions and videos captured in a multi-player social deduction game setting, 26,647 utterance level annotations of persuasion strategy, and game level annotations of deduction game outcomes. We provide extensive experiments to show how dialogue context and visual signals benefit persuasion strategy prediction. We also explore the generalization ability of language models for persuasion modeling and the role of persuasion strategies in predicting social deduction game outcomes. Our dataset, code, and models can be found at https://persuasion-deductiongame.socialai-data.org.

Unsupervised image registration commonly adopts U-Net style networks to predict dense displacement fields in the full-resolution spatial domain. For high-resolution volumetric image data, this process is however resource intensive and time-consuming. To tackle this problem, we propose the Fourier-Net, replacing the expansive path in a U-Net style network with a parameter-free model-driven decoder. Specifically, instead of our Fourier-Net learning to output a full-resolution displacement field in the spatial domain, we learn its low-dimensional representation in a band-limited Fourier domain. This representation is then decoded by our devised model-driven decoder (consisting of a zero padding layer and an inverse discrete Fourier transform layer) to the dense, full-resolution displacement field in the spatial domain. These changes allow our unsupervised Fourier-Net to contain fewer parameters and computational operations, resulting in faster inference speeds. Fourier-Net is then evaluated on two public 3D brain datasets against various state-of-the-art approaches. For example, when compared to a recent transformer-based method, i.e., TransMorph, our Fourier-Net, only using 0.22$\%$ of its parameters and 6.66$\%$ of the mult-adds, achieves a 0.6\% higher Dice score and an 11.48$\times$ faster inference speed. Code is available at \url{https://github.com/xi-jia/Fourier-Net}.

由于其极端的长距离建模能力，基于视觉变压器的网络在可变形图像注册中变得越来越流行。但是，我们认为，5层卷积U-NET的接受场足以捕获准确的变形而无需长期依赖性。因此，这项研究的目的是研究与现代变压器的方法相比，将基于U-NET的方法用于医学图像注册时是否已过时。为此，我们通过将平行的卷积块嵌入香草U-NET以增强有效的接受场来提出一个大核U-NET（LKU-NET）。在公共3D IXI Brain Dataset上，用于基于ATLAS的注册，我们表明，香草U-NET的性能已经与基于最新的变压器网络（例如Transmorph）相提并论，并且提出的LKU-NET仅使用其参数的1.12％和其多添加操作的10.8％，优于Transmorph。我们进一步评估了MICCAI Learn2Reg 2021挑战数据集中的LKU-NET，以进行主题间注册，我们的LKU-NET在此数据集中也优于TransMorph，并且在此工作提交后，在公共排行榜上排名第一。只有对香草U-NET的适度修改，我们表明U-NET可以在基于主体间和基于ATLAS的3D医疗图像注册上胜过基于变压器的体系结构。代码可在https://github.com/xi-jia/lku-net上找到。

我们介绍了一个新的问题，即从以自我为中心的视频中预期一个未来的手罩时间序列。一个关键的挑战是对未来头部运动的随机性进行建模，该动作在全球范围内影响了头饰的摄像头视频分析。为此，我们提出了一个新颖的深层生成模型-Egogan，它使用3D完全卷积网络来学习一个时空视频表示，以视觉预期，可以使用生成的对抗网络（GAN）和然后，根据视频表示和生成的未来头部运动来预测未来的手蒙版。我们在Epic-Kitchens和Egtea凝视+数据集上评估了我们的方法。我们进行详细的消融研究，以验证我们方法的设计选择。此外，我们将我们的方法与以前的未来图像分割方法进行比较，并表明我们的方法可以更准确地预测未来的手掩模。

最近，模型驱动的深度学习通过用网络模块替换符号器的一阶信息（即（子）梯度或近端运算符）来拓展到级联网络中的一定迭代算法，该算法呈现出更可说明的与常见的数据驱动网络相比，可以预测。相反，理论上，不一定存在这样的功能常规程序，其一级信息与替换的网络模块匹配，这意味着网络输出可能不被原始正则化模型覆盖。此外，到目前为止，在现实假设下，也没有保证展开网络的全球收敛性和鲁棒性（规律性）。为了弥合这一差距，本文建议在展开网络上提出保障方法。具体而言，专注于加速MRI，我们展开了一个零阶算法，网络模块代表常规器本身，使得网络输出可以仍然被正则化模型覆盖。此外，受到深度均衡模型的理想的启发，在反向化之前，我们执行了展开的迭代网络，以收敛到一个固定点，以确保收敛。如果测量数据包含噪声，我们证明了所提出的网络对嘈杂干扰具有强大。最后，数值实验表明，所提出的网络始终如一地优于最先进的MRI重建方法，包括传统的正规化方法和其他深度学习方法。

In this paper, we propose a robust 3D detector, named Cross Modal Transformer (CMT), for end-to-end 3D multi-modal detection. Without explicit view transformation, CMT takes the image and point clouds tokens as inputs and directly outputs accurate 3D bounding boxes. The spatial alignment of multi-modal tokens is performed implicitly, by encoding the 3D points into multi-modal features. The core design of CMT is quite simple while its performance is impressive. CMT obtains 73.0% NDS on nuScenes benchmark. Moreover, CMT has a strong robustness even if the LiDAR is missing. Code will be released at https://github.com/junjie18/CMT.

A recent study has shown a phenomenon called neural collapse in that the within-class means of features and the classifier weight vectors converge to the vertices of a simplex equiangular tight frame at the terminal phase of training for classification. In this paper, we explore the corresponding structures of the last-layer feature centers and classifiers in semantic segmentation. Based on our empirical and theoretical analysis, we point out that semantic segmentation naturally brings contextual correlation and imbalanced distribution among classes, which breaks the equiangular and maximally separated structure of neural collapse for both feature centers and classifiers. However, such a symmetric structure is beneficial to discrimination for the minor classes. To preserve these advantages, we introduce a regularizer on feature centers to encourage the network to learn features closer to the appealing structure in imbalanced semantic segmentation. Experimental results show that our method can bring significant improvements on both 2D and 3D semantic segmentation benchmarks. Moreover, our method ranks 1st and sets a new record (+6.8% mIoU) on the ScanNet200 test leaderboard. Code will be available at https://github.com/dvlab-research/Imbalanced-Learning.

Witnessing the impressive achievements of pre-training techniques on large-scale data in the field of computer vision and natural language processing, we wonder whether this idea could be adapted in a grab-and-go spirit, and mitigate the sample inefficiency problem for visuomotor driving. Given the highly dynamic and variant nature of the input, the visuomotor driving task inherently lacks view and translation invariance, and the visual input contains massive irrelevant information for decision making, resulting in predominant pre-training approaches from general vision less suitable for the autonomous driving task. To this end, we propose PPGeo (Policy Pre-training via Geometric modeling), an intuitive and straightforward fully self-supervised framework curated for the policy pretraining in visuomotor driving. We aim at learning policy representations as a powerful abstraction by modeling 3D geometric scenes on large-scale unlabeled and uncalibrated YouTube driving videos. The proposed PPGeo is performed in two stages to support effective self-supervised training. In the first stage, the geometric modeling framework generates pose and depth predictions simultaneously, with two consecutive frames as input. In the second stage, the visual encoder learns driving policy representation by predicting the future ego-motion and optimizing with the photometric error based on current visual observation only. As such, the pre-trained visual encoder is equipped with rich driving policy related representations and thereby competent for multiple visuomotor driving tasks. Extensive experiments covering a wide span of challenging scenarios have demonstrated the superiority of our proposed approach, where improvements range from 2% to even over 100% with very limited data. Code and models will be available at https://github.com/OpenDriveLab/PPGeo.

This paper illustrates the technologies of user next intent prediction with a concept knowledge graph. The system has been deployed on the Web at Alipay, serving more than 100 million daily active users. Specifically, we propose AlipayKG to explicitly characterize user intent, which is an offline concept knowledge graph in the Life-Service domain modeling the historical behaviors of users, the rich content interacted by users and the relations between them. We further introduce a Transformer-based model which integrates expert rules from the knowledge graph to infer the online user's next intent. Experimental results demonstrate that the proposed system can effectively enhance the performance of the downstream tasks while retaining explainability.

Human parsing aims to partition humans in image or video into multiple pixel-level semantic parts. In the last decade, it has gained significantly increased interest in the computer vision community and has been utilized in a broad range of practical applications, from security monitoring, to social media, to visual special effects, just to name a few. Although deep learning-based human parsing solutions have made remarkable achievements, many important concepts, existing challenges, and potential research directions are still confusing. In this survey, we comprehensively review three core sub-tasks: single human parsing, multiple human parsing, and video human parsing, by introducing their respective task settings, background concepts, relevant problems and applications, representative literature, and datasets. We also present quantitative performance comparisons of the reviewed methods on benchmark datasets. Additionally, to promote sustainable development of the community, we put forward a transformer-based human parsing framework, providing a high-performance baseline for follow-up research through universal, concise, and extensible solutions. Finally, we point out a set of under-investigated open issues in this field and suggest new directions for future study. We also provide a regularly updated project page, to continuously track recent developments in this fast-advancing field: https://github.com/soeaver/awesome-human-parsing.